System and method for facilitating user interaction with a simulated object associated with a physical location

ABSTRACT

Systems and methods for facilitating user interaction with a simulated object that is associated with a physical location in the real world environment is herein disclosed. In one aspect, embodiments of the present disclosure include a method, which may be implemented on a system, of identifying the simulated object that is available for access based on location data. The location data can include a location of a device in a time period, the device for use by a user to access the simulated object. One embodiment includes, verifying an identity of the user; and in response to determining that the user is authorized to access the simulated object, providing the simulated object for presentation to the user via the device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of prior application Ser.No. 14/826,123, filed on Aug. 13, 2015, which is a continuationapplication of prior application Ser. No. 12/473,171 filed on May 27,2009, the disclosure of each of which is incorporated by referenceherein in its entirety.

TECHNICAL FIELD

This technology relates generally to virtual reality and in particular,to virtual realities representing and associated with a physicallocation and applications thereof.

BACKGROUND

Miniaturization of consumer electronics with sophisticated graphicscapabilities and expansive computing power has augmented the activitiesone can engage in via consumer electronics and in particular, portableelectronics such as cell phones, PDAs, Blackberries, iPhones, and thelike.

Further, portable electronics or other electronics devices now generallyinclude GPS or other types of location sensing capabilities. Thus,mobile application capabilities and user experiences can be enhancedwith the awareness of location information, such as location data thatincludes the real time or current location of the user or the device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example block diagram of client devices able tocommunicate with a host server that generates and controls access tosimulated objects through a network.

FIG. 2 depicts an example block diagram of the components of a hostserver that generates and controls simulated objects.

FIG. 3A depicts an example functional block diagram of the host serverthat generates and controls access to simulated objects.

FIG. 3B depicts an example block diagram illustrating the components ofthe host server that generates and controls access to simulated objects.

FIG. 4A depicts an example functional block diagram of a client devicethat presents simulated objects to a user and processes interactionswith the simulated objects.

FIG. 4B depicts an example block diagram of the client device thatpresents simulated objects to a user and facilitates user interactionswith the simulated objects.

FIG. 5A illustrates a diagrammatic example of a simulated playing fieldthat is provided via a device.

FIG. 5B illustrates a diagrammatic example of virtual performances witha simulated object that is controlled by a real performer.

FIG. 5C illustrates an example screenshot on a device displaying asimulated environment with a simulated object associated with a physicalobject in a physical location in the real world environment.

FIG. 5D illustrates a diagrammatic example of an arcade game in a gamingenvironment that corresponds to a physical location and real players ina real world environment.

FIG. 5E illustrates a diagrammatic example of a virtual game having asimulated combat environment that is played by a real user in a realworld environment via a device.

FIG. 5F illustrates a diagrammatic example of a simulated objectrepresenting an interactive puzzle or a component thereof.

FIG. 5G illustrates a diagrammatic example of simulated objects thatrepresent real-time or near-real time information/data projected ontogeographical locations in a map.

FIG. 6 depicts a flow chart illustrating an example process fortime-based control/manipulation of a simulated object that is associatedwith a physical location in a real world environment.

FIG. 7A depicts a flow chart illustrating an example process forfacilitating user interaction with a simulated object that is associatedwith a physical location in a real world environment.

FIG. 7B depicts a flow chart illustrating example processes for updatingthe simulated object and the simulated environment according to externalstimulus.

FIG. 8 depicts a flow chart illustrating an example process forsimulating a virtual sports game played by a real participant in a realworld environment.

FIG. 9 depicts a flow chart illustrating an example process forsimulating a virtual game played by a real user in a real worldenvironment.

FIG. 10 a flow chart illustrating an example process for simulating avirtual performance in a real world environment.

FIG. 11 shows a diagrammatic representation of a machine in the exampleform of a computer system within which a set of instructions, forcausing the machine to perform any one or more of the methodologiesdiscussed herein, may be executed, according to one embodiment.

DETAILED DESCRIPTION

The following description and drawings are illustrative and are not tobe construed as limiting. Numerous specific details are described toprovide a thorough understanding of the disclosure. However, in certaininstances, well-known or conventional details are not described in orderto avoid obscuring the description. References to one or an embodimentin the present disclosure can be, but not necessarily are, references tothe same embodiment; and, such references mean at least one of theembodiments.

Reference in this specification to “one embodiment” or “an embodiment”means that a particular feature, structure, or characteristic describedin connection with the embodiment is included in at least one embodimentof the disclosure. The appearances of the phrase “in one embodiment” invarious places in the specification are not necessarily all referring tothe same embodiment, nor are separate or alternative embodimentsmutually exclusive of other embodiments. Moreover, various features aredescribed which may be exhibited by some embodiments and not by others.Similarly, various requirements are described which may be requirementsfor some embodiments but not other embodiments.

The terms used in this specification generally have their ordinarymeanings in the art, within the context of the disclosure, and in thespecific context where each term is used. Certain terms that are used todescribe the disclosure are discussed below, or elsewhere in thespecification, to provide additional guidance to the practitionerregarding the description of the disclosure. For convenience, certainterms may be highlighted, for example using italics and/or quotationmarks. The use of highlighting has no influence on the scope and meaningof a term; the scope and meaning of a term is the same, in the samecontext, whether or not it is highlighted. It will be appreciated thatsame thing can be said in more than one way.

Consequently, alternative language and synonyms may be used for any oneor more of the terms discussed herein, nor is any special significanceto be placed upon whether or not a term is elaborated or discussedherein. Synonyms for certain terms are provided. A recital of one ormore synonyms does not exclude the use of other synonyms. The use ofexamples anywhere in this specification including examples of any termsdiscussed herein is illustrative only, and is not intended to furtherlimit the scope and meaning of the disclosure or of any exemplifiedterm. Likewise, the disclosure is not limited to various embodimentsgiven in this specification.

Without intent to further limit the scope of the disclosure, examples ofinstruments, apparatus, methods and their related results according tothe embodiments of the present disclosure are given below. Note thattitles or subtitles may be used in the examples for convenience of areader, which in no way should limit the scope of the disclosure. Unlessotherwise defined, all technical and scientific terms used herein havethe same meaning as commonly understood by one of ordinary skill in theart to which this disclosure pertains. In the case of conflict, thepresent document, including definitions will control.

Embodiments of the present disclosure include systems and methods forfacilitating user interaction with a simulated object that is associatedwith a physical location in the real world environment.

FIG. 1 illustrates an example block diagram of client devices 102A-Nable to communicate with a host server 124 that generates and controlsaccess to simulated objects through a network 106.

The client devices 102A-N can be any system and/or device, and/or anycombination of devices/systems that is able to establish a connectionwith another device, a server and/or other systems. The client devices102A-N typically include a display and/oror other output functionalitiesto present information and data exchanged between among the devices102A-N and the host server 124. For example, the client devices 102A-Ncan be any of, but are not limited to, a server desktop, a desktopcomputer, a computer cluster, or portable devices including, a notebook,a laptop computer, a handheld computer, a palmtop computer, a mobilephone, a cell phone, a smart phone, a PDA, a Blackberry device, a Treo,an iPhone, cover headsets, heads-up displays, helmet mounted display,head-mounted display, scanned-beam display, wearable computer such asmobile enabled watches, and/or any other mobile interfaces and viewingdevices, etc. The client devices 102A-N may be location-aware devicesthat are able to determine their own location or identify locationinformation from an external source. In one embodiment, the clientdevices 102A-N are coupled to a network 106. In some embodiments, thedevices 102A-N and host server 124 may be directly connected to oneanother.

In one embodiment, the host server 124 is operable to provide simulatedobjects (e.g., objects, computer-controlled objects, or simulatedobjects) that correspond to real world physical locations to bepresented to users on client devices 102A-N. The simulated objects aretypically software entities or occurrences that are controlled bycomputer programs and can be generated upon request when certaincriteria are met. The host server 124 also processes interactions ofsimulated object with one another and actions on simulated objectscaused by stimulus from a real user and/or the real world environment.Services and functions provided by the host server 124 and thecomponents therein are described in detail with further references tothe examples of FIG. 3A-3B.

The client devices 102A-N are generally operable to provide access(e.g., visible access, audible access) to the simulated objects tousers, for example via user interface 104A-N displayed on the displayunits. The devices 102A-N may be able to detect simulated objects basedon location and/or timing data and provide those objects authorized bythe user for access via the devices. Services and functions provided bythe client devices 102A-N and the components therein are described indetail with further references to the examples of FIG. 4A-4B.

The network 106, over which the client devices 102A-N and the hostserver 124 communicate, may be a telephonic network, an open network,such as the Internet, or a private network, such as an intranet and/orthe extranet. For example, the Internet can provide file transfer,remote log in, email, news, RSS, and other services through any known orconvenient protocol, such as, but is not limited to the TCP/IP protocol,Open System Interconnections (OSI), FTP, UPnP, iSCSI, NSF, ISDN, PDH,RS-232, SDH, SONET, etc.

The network 106 can be any collection of distinct networks operatingwholly or partially in conjunction to provide connectivity to the clientdevices 102A-N and the host server 124 and may appear as one or morenetworks to the serviced systems and devices. In one embodiment,communications to and from the client devices 102A-N can be achieved by,an open network, such as the Internet, or a private network, such as anintranet and/or the extranet. In one embodiment, communications can beachieved by a secure communications protocol, such as secure socketslayer (SSL), or transport layer security (TLS).

In addition, communications can be achieved via one or more wirelessnetworks, such as, but is not limited to, one or more of a Local AreaNetwork (LAN), Wireless Local Area Network (WLAN), a Personal areanetwork (PAN), a Campus area network (CAN), a Metropolitan area network(MAN), a Wide area network (WAN), a Wireless wide area network (WWAN),Global System for Mobile Communications (GSM), Personal CommunicationsService (PCS), Digital Advanced Mobile Phone Service (D-Amps),Bluetooth, Wi-Fi, Fixed Wireless Data, 2G, 2.5G, 3G networks, enhanceddata rates for GSM evolution (EDGE), General packet radio service(GPRS), enhanced GPRS, messaging protocols such as, TCP/IP, SMS, MMS,extensible messaging and presence protocol (XMPP), real time messagingprotocol (RTMP), instant messaging and presence protocol (IMPP), instantmessaging, USSD, IRC, or any other wireless data networks or messagingprotocols.

The host server 124 may include or be coupled to a user repository 128and/or a simulated object repository 130. The user data repository 128can store software, descriptive data, images, system information,drivers, and/or any other data item utilized by other components of thehost server 124 and/or any other servers for operation. The user datarepository 128 may be managed by a database management system (DBMS),for example but not limited to, Oracle, DB2, Microsoft Access, MicrosoftSQL Server, PostgreSQL, MySQL, FileMaker, etc.

The user data repository 128 and/or the simulated object repository 130can be implemented via object-oriented technology and/or via text files,and can be managed by a distributed database management system, anobject-oriented database management system (OODBMS) (e.g., ConceptBase,FastDB Main Memory Database Management System, JDOInstruments, ObjectDB,etc.), an object-relational database management system (ORDBMS) (e.g.,Informix, OpenLink Virtuoso, VMDS, etc.), a file system, and/or anyother convenient or known database management package.

In some embodiments, the host server 124 is able to provide data to bestored in the user data repository 128 and/or the simulated objectrepository 130 and/or can retrieve data stored in the user datarepository 128 and/or the simulated object repository 130. The user datarepository 128 can store user information, user preferences, accesspermissions associated with the users, device information, hardwareinformation, etc. The simulated object repository 130 can store softwareentities (e.g., computer programs) that control simulated objects andthe simulated environments in which they are presented forvisual/audible access or control/manipulation. The simulated objectrepository 130 may further include simulated objects and theirassociated data structures with metadata defining the simulated objectincluding its associated access permission.

FIG. 2 depicts an example block diagram of the components of a hostserver 224 that generates and controls simulated objects.

In the example of FIG. 2, the host server 224 includes a networkcontroller 202, a firewall 204, a multimedia server 206, an applicationserver 208, a web application server 212, a gaming server 214, and adatabase including a database storage 216 and database software 218.

In the example of FIG. 2, the network controller 202 can be a networkingdevice that enables the host server 224 to mediate data in a networkwith an entity that is external to the host server 224, through anyknown and/or convenient communications protocol supported by the hostand the external entity. The network controller 202 can include one ormore of a network adaptor card, a wireless network interface card, arouter, an access point, a wireless router, a switch, a multilayerswitch, a protocol converter, a gateway, a bridge, bridge router, a hub,a digital media receiver, and/or a repeater.

The firewall 204, can, in some embodiments, govern and/or managepermission to access/proxy data in a computer network, and track varyinglevels of trust between different machines and/or applications. Thefirewall 204 can be any number of modules having any combination ofhardware and/or software components able to enforce a predetermined setof access rights between a particular set of machines and applications,machines and machines, and/or applications and applications, forexample, to regulate the flow of traffic and resource sharing betweenthese varying entities. The firewall 204 may additionally manage and/orhave access to an access control list which details permissionsincluding for example, the access and operation rights of an object byan individual, a machine, and/or an application, and the circumstancesunder which the permission rights stand.

Other network security functions can be performed or included in thefunctions of the firewall 204, can be, for example, but are not limitedto, intrusion-prevention, intrusion detection, next-generation firewall,personal firewall, etc. without deviating from the novel art of thisdisclosure. In some embodiments, the functionalities of the networkcontroller 202 and the firewall 204 are partially or wholly combined andthe functions of which can be implemented in any combination of softwareand/or hardware, in part or in whole.

In the example of FIG. 2, the host server 200 includes the multimediaserver 206 or a combination of multimedia servers to manage images,photographs, animation, video, audio content, graphical content,documents, and/or other types of multimedia data for use in or tosupplement simulated content such as simulated objects and theirassociated deployment environment (e.g., a simulated environment). Themultimedia server 206 is any software suitable for delivering messagesto facilitate retrieval/transmission of multimedia data among servers tobe provided to other components and/or systems of the host server 224,for example when rendering a web page, a simulated environment, and/orsimulated objects including multimedia content.

In addition, the multimedia server 206 can facilitatetransmission/receipt of streaming data such as streaming images, audio,and/or video. The multimedia server 206 can be configured separately ortogether with the web application server 212, depending on a desiredscalability of the host server 224. Examples of graphics file formatsthat can be managed by the multimedia server 206 include but are notlimited to, ADRG, ADRI, AI, GIF, IMA, GS, JPG, JP2, PNG, PSD, PSP, TIFF,and/or BMP, etc.

The application server 208 can be any combination of software agentsand/or hardware modules for providing software applications to endusers, external systems and/or devices. For example, the applicationserver 208 provides specialized or generic software applications thatmanage simulated environments and objects to devices (e.g., clientdevices). The software applications provided by the application server208 can be automatically downloaded on-demand on an as-needed basis ormanually at the user's request. The software applications, for example,allow the devices to detect simulated objects based on the location ofthe device and to provide the simulated objects for access, based onpermissions associated with the user and/or with the simulated object.

Additionally, nearby users or players can be also be automaticallydetected. The detected users/players can be represented on the userdevice for example, as a simulated object controlled by the nearbyusers/players. In addition, simulated objects having a particular set oftemporal/spatial attributes may be detected by the user device. Thesimulated objects may or may not represent real-life users. The softwareapplications provided by the application server 208 can be used by usersto access, manipulate, and/or control simulated objects using theirdevices. Additional details related to the functions of the softwareapplications are described with further reference to the example of FIG.4A-B.

The application server 208 can also facilitate interaction andcommunication with the web application server 212, or with other relatedapplications and/or systems. The application server 208 can in someinstances, be wholly or partially functionally integrated with the webapplication server 212.

The web application server 212 can include any combination of softwareagents and/or hardware modules for accepting Hypertext Transfer Protocol(HTTP) requests from end users, external systems, and/or external clientdevices and responding to the request by providing the requestors withweb pages, such as HTML documents and objects that can include staticand/or dynamic content (e.g., via one or more supported interfaces, suchas the Common Gateway Interface (CGI), Simple CGI (SCGI), PHP,JavaServer Pages (JSP), Active Server Pages (ASP), ASP.NET, etc.).

In addition, a secure connection, SSL and/or TLS can be established bythe web application server 212. In some embodiments, the web applicationserver 212 renders the user interfaces having the simulated environmentas shown in the example screenshots of FIG. 5A-FIG. 5C. The userinterfaces provided by the web application server 212 to clientusers/end devices provide the user interface screens 104A-104N forexample, to be displayed on client devices 102A-102N. In someembodiments, the web application server 212 also performs anauthentication process before responding to requests for access,control, and/or manipulation of simulated objects and simulatedenvironments.

In one embodiment, the host server 200 includes a gaming server 214including software agents and/or hardware modules for providing gamesand gaming software to client devices. The games and gaming environmentstypically include simulations of real world environments. The gamingserver 214 also provides games and gaming environments such that thesimulated objects provided therein have characteristics that areaffected and can be manipulated by external stimuli (e.g., stimuli thatoccur in the real world environment) and can also interact with othersimulated objects. External stimuli can include real physical motion ofthe user, motion of the device, user interaction with the simulatedobject on the device, and/or real world environmental factors, etc.

For example, the external stimuli detected at a client device may beconverted to a signal and transmitted to the gaming server 214. Thegaming server 214, based on the signal, updates the simulated objectand/or the simulated environment such that a user of the client deviceperceives such changes to the simulated environment in response to realworld stimulus. The gaming server 214 provides support for any type ofsingle player or multiplayer electronic gaming, PC gaming, arcadegaming, and/or console gaming for portable devices or non-portabledevices. These games typically have real world location correlatedfeatures and may have time or user constraints on accessibility,availability, and/or functionality. The objects simulated by the gamingserver 214 are presented to users via devices and can be controlledand/or manipulated by authorized users.

The databases 216, 218 can store software, descriptive data, images,system information, drivers, and/or any other data item utilized byother components of the host server for operation. The databases 216,218 may be managed by a database management system (DBMS), for examplebut not limited to, Oracle, DB2, Microsoft Access, Microsoft SQL Server,PostgreSQL, MySQL, FileMaker, etc. The databases 216, 218 can beimplemented via object-oriented technology and/or via text files, andcan be managed by a distributed database management system, anobject-oriented database management system (OODBMS) (e.g., ConceptBase,FastDB Main Memory Database Management System, JDOInstruments, ObjectDB,etc.), an object-relational database management system (ORDBMS) (e.g.,Informix, OpenLink Virtuoso, VMDS, etc.), a file system, and/or anyother convenient or known database management package.

In the example of FIG. 2, the host server 200 includes components (e.g.,a network controller, a firewall, a storage server, an applicationserver, a web application server, a gaming server, and/or a databaseincluding a database storage and database software, etc.) coupled to oneanother and each component is illustrated as being individual anddistinct. However, in some embodiments, some or all of the components,and/or the functions represented by each of the components can becombined in any convenient or known manner. Furthermore, the functionsrepresented by the devices can be implemented individually or in anycombination thereof, in hardware, software, or a combination of hardwareand software.

FIG. 3A depicts an example functional block diagram of the host server324 that generates and controls access to simulated objects.

The host server 324 includes a network interface 302, a simulator module304, an environment simulator module 306, a virtual sports simulator308, a virtual game simulator 310, a virtual performance simulator 312,an access permission module 314, an interactions manager module 316, anenvironmental factor sensor module 318, an object control module 320,and/or a search engine 322. In one embodiment, the host server 224 iscoupled to a user data repository 328 and/or a simulated objectrepository 330. The user data repository 328 and simulated objectrepository 330 are described with further reference to the example ofFIG. 1.

Additional or less modules can be included without deviating from thenovel art of this disclosure. In addition, each module in the example ofFIG. 3A can include any number and combination of sub-modules, andsystems, implemented with any combination of hardware and/or softwaremodules.

The host server 324, although illustrated as comprised of distributedcomponents (physically distributed and/or functionally distributed),could be implemented as a collective element. In some embodiments, someor all of the modules, and/or the functions represented by each of themodules can be combined in any convenient or known manner. Furthermore,the functions represented by the modules can be implemented individuallyor in any combination thereof, partially or wholly, in hardware,software, or a combination of hardware and software.

In the example of FIG. 3A, the network interface 302 can be a networkingdevice that enables the host server 324 to mediate data in a networkwith an entity that is external to the host server, through any knownand/or convenient communications protocol supported by the host and theexternal entity. The network interface 302 can include one or more of anetwork adaptor card, a wireless network interface card, a router, anaccess point, a wireless router, a switch, a multilayer switch, aprotocol converter, a gateway, a bridge, bridge router, a hub, a digitalmedia receiver, and/or a repeater.

One embodiment of the host server 324 includes a simulator module 304.The simulator module 304 can be any combination of software agentsand/or hardware modules able to create, generate, modify, update,adjust, edit, and/or delete a simulated object.

A simulated object (or, otherwise referred as, a software object, acomputer-controlled object, a simulated object, an object, etc.)typically refers to a software entity/software controlled entity that iscontrolled by a computer program. A simulated object can include asimulation of a physical entity, a concept/idea, an imaginary entity, asoftware object, an occurrence, an event, a living object, an inanimateobject, and/or a real or imaginary phenomenon/object with strong,partial, or no resemblance to the physical appearances, observableproperties of these entities. Simulated objects can be provided for ordeployed in various types of simulated environments alsocontrolled/managed by software.

Characteristics and attributes of simulated objects can be perceived byusers in reality via a physical device (e.g., a client device or device102 in the example of FIG. 1). For example, a simulated object typicallyincludes visible and/or audible characteristics that can be perceived byusers via a device with a display and/or a speaker. Changes tocharacteristics and attributes of simulated objects can also beperceived by users in reality via physical devices.

In one embodiment, these simulated objects are associated with physicallocations in the real world environment and have associatedaccessibilities based on a spatial parameter (e.g., the location of adevice through which the simulated object is to be accessed). In someinstances, the simulated objects have associated accessibilities basedon a temporal parameter as well as user-specificities (e.g., certainusers may be different access rights to different simulated objects).

Objects may be simulated by the simulator module 304 automatically ormanually based on a user request. For example, objects may be simulatedautomatically when certain criterion (e.g., qualifying location dataand/or qualifying timing data) are met or upon request by anapplication. Objects may also be newly created/simulated when anauthorized user requests objects that are not yet available (e.g.,object is not stored in the simulated object repository 330). Generatedobjects can be stored in the simulated object repository 330 for futureuse.

In one embodiment, the simulated object is implemented using a datastructure having metadata. The metadata can include a computer programthat controls the actions/behavior/properties of the simulated objectand how behaviors of the simulated object are affected by a user orother external factors (e.g., real world environmental factors). Themetadata can also include location and/or timing parameters that includethe qualifying parameters (e.g., qualifying timing and/or location data)that satisfy one or more criteria for access of the simulated object tobe enabled. The location data can be specified with longitude andlatitude coordinates, GPS coordinates, and/or relative position. In oneembodiment, the object is associated with a unique identifier. Theunique identifier may be further associated with a location datastructure having a set of location data that includes the qualifyinglocation data for the simulated object.

The metadata can include different criteria for different types ofaccess of the simulated object. The different types of accessibility caninclude, create, read, view, write, modify, edit, delete, manipulate,and/or control etc. Each of these actions can be associated with adifferent criterion that is specified in the object's metadata. Inaddition to having temporal and spatial parameters, some criterion mayalso include user-dependent parameters. For example, certain users haveedit right where other users only have read/viewing rights. These rightsmay be stored as user access permissions associated with the user orstored as object access permission rights associated with the simulatedobject. In one embodiment, the metadata includes a link to anothersimulated object and/or data from an external source (e.g., theInternet, Web, a database, etc.). The link may be a semantic link.

One embodiment of the host server 324 includes an environment simulatormodule 306. The environment simulator module 306 can be any combinationof software agents and/or hardware modules able to generate, modify,update, adjust, and/or delete a simulated environment in which simulatedobjects are presented.

In one embodiment, the simulated environment is associated with aphysical location in the real world environment. The simulatedenvironment thus may include characteristics that correspond to thephysical characteristics of the associated physical location. Oneembodiment of the host server 224 includes the environment simulatormodule 306 which may be coupled to the simulator module 304 and canrender simulated environments in which the simulated object is deployed.

The simulated objects are typically visually provided in the simulatedenvironment for display on a device display. Note that the simulatedenvironment can include various types of environments including but notlimited to, a gaming environment, a virtual sports environment, avirtual performance environment, a virtual teaching environment, avirtual indoors/outdoors environment, a virtual underwater environment,a virtual airborne environment, a virtual emergency environment, avirtual working environment, and/or a virtual tour environment.

For example, in a simulated environment with a virtual concert that isvisible to the user using a device, the simulated objects in the virtualconcert may include those controlled by a real musician (e.g. recordedor in real time). Other simulated objects in the virtual concert mayfurther include simulated instruments with audible characteristics suchas sound played by the real instruments that are represented by thesimulated instruments. Additional simulated objects may be provided inthe virtual concert for decorative purposes and/or to provide thefeeling that one is in a real concert. For example, additional simulatedobjects may include a simulated audience, a simulated applause, etc.

In one example, the simulated environment is associated with a physicallocation that is a tourist location in the real world environment. Thesimulated object associated with the tourist location can include videoand audio data about the tourist location. The audio data can includecommentary about the historical value of the site. The simulated objectmay also include a link to other simulated objects corresponding toother nearby tourist attractions or sites and serve as a self-servetravel guide or personal travel agent.

In one embodiment, this information is automatically provided to theuser when he or she arrives at or near the real world tourist location(e.g., implicit request) via the device. Alternatively, the informationis provided upon request by the user (e.g., explicit request). Forexample, simulated objects associated with various attractions in thetourist location in the real world can be selected by the user (e.g.,via input to the device). The simulated objects that are selected mayperform playback of the textual, video and/or audio data about theattractions in the real world tourist location.

In one example, the simulated object is an advertisement (e.g., anelectronic advertisement) and the user to whom the simulated object ispresented is a qualified user targeted by the advertisement. The usermay qualify on a basis of a location, identity, and/or a timingparameter. For example, the user may be provided with advertisements oflocal pizza shops or other late night dining options when the user isdriving around town during late night hours when other dining optionsmay not be available.

In one example, the simulated environment is used for education andtraining of emergency services providers and/or law enforcementindividuals. These simulated environments may include virtual drillswith simulated objects that represent medical emergencies or hostages.The users that access these simulated virtual drills may include medicalservice providers, firefighters, and/or law enforcers.

In a further example, simulated objects can represent electronicdocuments (e.g., files or datasets) that are visible using the devicewhen the device is in a particular physical location in the real worldenvironment. For example, a document or note can be left for a user at asimulated location that corresponds to a real world location. In oneembodiment, the simulated object represents an electronic document andthe user retrieves the electronic document using the device when thelocation of the device satisfies a criteria. For example, the electronicdocument is a reference manual for a physical object and can beaccessible to the user when the location of the device is within a rangeof the physical object.

In another example, simulated objects with access permissions that onspatial and temporal parameters can be used to data protection. Thesimulated object that represents the protected data may only be viewedusing devices located at an authorized location or in an authorizedfacility. The user viewing the protected data may also be an authorizeduser. Thus, the protected data cannot be viewed by anyone outside theauthorized location/facility or by anyone that is not authorized. Theprotected data may only be viewed during a certain period of time.

In one example, the simulated environment is a virtual desktop thatincludes simulated objects. The simulated objects may be associated withreal physical locations near a user and be placed in space relative tothe user. In one embodiment, access to the simulated objects may beenabled for those associated with the real physical locations visiblethrough an imaging unit of the device (e.g., a camera in a cell phone orPDA). For example, when a user views physical space with a camera on acell phone, the user can see the simulated objects in the virtualdesktop displayed on the cell phone. The virtual desktop appears to theuser as if it is in the surrounding space and may include features thatcorrespond to the real surrounding space. The device can be moved inspace such that different simulated objects associated with differentphysical locations are imaged through the cell phone camera and thusaccessed.

In another example, a simulated environment can be used for taskmanagement. For example, the simulated object can represent or includeinformation related to a task. The simulated tasks can be presented tothe user through the device when located at or near the location wherethe task is to be performed. For example, information about deliveriescan be placed for a driver at various real world delivery locations.Thus, the driver can be notified of this information on their deviceswhen they arrive at the delivery locations. The information morerelevant to their present location can be displayed as more visible orprominent with higher priority in the user interface displayed on thedevice.

In one embodiment, the simulated object is a virtual personal assistantof the user. The virtual personal assistant can be pre-programmed orconfigured to follow the user around as they move around in realphysical space. The virtual personal assistant may be visible to theuser via the device anywhere they go. The virtual personal assistancemay also be visible to others via devices with access permissions.

The simulated environment may be a virtual marketplace associated withthe physical location in the real world environment. The simulatedobjects and represent either real goods or virtual goods for users tosell or purchase when the device is located in the physical locationassociated with the virtual market place. In general, users with adevice with the appropriate software capabilities and/or proper accesspermissions can see the simulated objects and buy or sell thecorresponding goods.

In one embodiment, the simulated object represents an electronic couponand is accessible to a user using the device when the device is locatedat the location during a certain period of time that satisfies thecriteria. The electronic coupon may be redeemed by the user at abusiness located at or near the location in the real world environment.

One embodiment of the host server 324 includes an access permissionmodule 314. The access permission module 314 can be any combination ofsoftware agents and/or hardware modules able to determine availabilityand accessibility of a simulated object based on a criterion.

The criteria can include spatio-temporal criteria having a timingparameter and/or a location parameter. For example, a simulated objectmay be associated with a physical location in the real worldenvironment. The location parameter may include a set of locationsincluding the physical location and/or surrounding regions where thedevice is to be located to access the simulated object. In addition, thetiming parameter includes a time or set of times when the simulatedobject can be accessed. The timing parameter and the location parametercan be used independently or in conjunction with each other.

The access permission module 314 can determine whether a location dataand/or a timing data satisfy the criterion (e.g., a spatio-temporalcriterion). The access permission module 314 is coupled to the simulatormodule 304, the environment simulator module 306, and the simulatedobject repository 330, where simulated objects and/or simulatedenvironments are stored. When the access permission module 314determines that the location and/or timing data satisfy the criterion,the access permission module 314 access of the simulated object in asimulated environment by a user via a device (e.g., portable ornon-portable device). One embodiment of the access permission module 314includes a timing module and a location sensor to determine the currenttime and/or the current location of a device.

In one embodiment, location data and/or the timing data that satisfy thecriterion include the location of the device and the time the device islocated at the location. The enable signal may be sent to the simulatorand environmental simulator modules such that the simulator module 304an enable access to the simulated object via a device when the criteriais met. The access permission module 314 may retrieve the relevantsimulated objects and simulated environments from other modules to beprovided to a user via a device.

In one embodiment, the access permission module 314 determines thecriterion associated with the simulated objects, for example, byretrieving and/or identifying metadata stored in the data structure ofthe simulated object that specifies qualifying timing data and/orqualifying location data that satisfy the criteria for object access. Inaddition, the access permission module 314 can set the access criteriafor a simulated object. For example, the access permission module 314can identify metadata of the simulated object and determine variousattributes of the simulated object to set some access criteria.

The access permission module 314 can also identify the user accesspermission associated with a particular user. For example, the accesspermission module 314 can retrieve user information from the userrepository 328. The user repository can be coupled to the simulatedobject repository 330 and can have stored therein access permissionsassociated with the user. Thus, the criterion to access a simulatedobject can further include a user-dependent parameter.

One embodiment of the host server 324 includes an interactions managermodule 316. The interactions manager module 316 can be any combinationof software agents and/or hardware modules able to monitor, manage,control user interactions and user requested interactions with thesimulated objects, and interactions among simulated objects.

The interactions manager module 316 can be coupled to the accesspermission module 314 to determine the criteria for interacting with thesimulated objects and whether the requesting user has permission toperform such requested actions on the simulated objects. Theinteractions manager module 316, upon receiving a request from the userto perform a requested action on the simulated object, the managermodule 316 determines whether the user is permitted to perform therequested action on the simulated object.

The interactions manager module 316 can identify this informationaccording to either user access permissions and/or object accesspermissions. The requested action is typically triggered by the user viathe device (e.g., portable device, location-aware device, PDA, cellphone, laptop, etc.) using input control (e.g., keyboard, mouse,joystick, pointing device, touch screen sensor, etc.) of the device.

If the user is permitted to interact with the simulated object, themanager module 316 can perform the requested action on the simulatedobject by updating stored attributes of the simulated objects andpresenting the updated attributes via the device to be perceived by theuser. In one embodiment, the simulator module 304 updates the attributesaccording to the requested action upon receiving the commands orsignals. The user requested actions can include, by way of example butnot limitation, collecting an item (e.g., a reward), firing ammunition,throwing an item, eating an item, attending an event, dialoguing withanother character (real or virtual), surmounting a barrier, hitting aball, blocking a ball, kicking a ball, and/or shooting a goblin, etc.These actions may be requested by the user using an input device or acombination of input devices.

Note that user actions requested with regards to simulated objects canbe stored, for later access or to compute statistics regarding usage,likeability, user preference, etc. User actions requested pertaining tosimulated objects an include one or more of, adding as a favorite andcollecting as a bookmark, sharing the simulated object, flagging thesimulated object, and/or tagging the simulated object. Additionally,user-generated data for simulated objects can also be recorded andstored. User-generated data an include, one or more of, modification ofthe simulated object, comment on the simulated object, review of thesimulated object, and/or rating of the simulated object. In someembodiments, the user modifies the simulated object using the device oranother device. In addition, the user can create or author the simulatedobject using any device.

Simulated objects may interact with one another. The interactionsmanager module 316 can control these interactions according to thecomputer programs that control the simulated objects. The simulatedobjects that interact with one another may be controlled/manipulated byreal users and/or wholly/partially controlled by computer programs.

One embodiment of the host server 324 includes an environmental sensormodule 318. The environmental sensor module 318 can be any combinationof software agents and/or hardware modules able to detect, sense,monitor, identify, track, and/or process environmental factors, physicalcharacteristics and changes that occur in the real world environment.

Since simulated environments can be sometimes thought not alwaysgenerated to correspond to simulation of a physical location in a realworld environment and/or regions proximal to the physical location, theenvironmental sensor module 318 can detect and sense the environmentalfactors and physical characteristics in the real world to facilitatesuch interactions. The environmental sensor module 318 is coupled to theenvironment simulator module 306 and can provide such information to theenvironmental simulator module 306 such that simulated environments,when generated, will correspond to simulation of the physical locationand regions proximal to the physical location.

In one embodiment, simulated objects and their associatedcharacteristics depend on stimuli that occur in the real worldenvironment. For example, the external stimuli that can change/affectbehaviors or appearances of a simulated object include environmentalfactors in or near the physical location associated with the simulatedobject. The environmental sensor module 318 can detect theseenvironmental factors and changes and communicate the information to thesimulator module 304 and/or the environmental simulator module 306 toimplement the effects of the environmental factors on the simulatedobject in software for presentation via devices.

The environmental factors detected by the environmental sensor module318 can include, by way of example but not limitation, temperature,weather, landscape, surrounding people, cars, animals, climate,altitude, topology, population, etc.

One embodiment of the host server 324 includes an object control module320. The object control module 320 can be any combination of softwareagents and/or hardware modules able to manage the control of simulatedobjects by real users in the real world environment.

Simulated objects, in addition to being manipulated and interacted withby users, can also be “controlled” by users. In a simulated environment,there may be simulated objects some of which are controlled by differentusers in different physical locations, for example. Control of asimulated object by a user can be defined more broadly than manipulationof or interaction with a simulated object. For example, the movements,behaviors, and/or actions of a simulated object can be simulations ofmovement, behaviors, and/or actions of a real user.

The movement trajectory of the simulated object in a simulatedenvironment, when controlled by a user, can be predominantly governed bymovement or behavior of the user. In a further example, the form/shapeof the simulated object may also depend on the physical appearances ofthe users. In addition, the simulated object may include audiblecharacteristics that depend on the user's voice or speech.

The object control module 320 determines permissions of users to controlthe simulated object. Changes to attributes of the simulated objectcaused by user control can be reflected in the simulated environment andperceived by the same controlling user or other users via a device. Thisupdate can occur with a delay or in real-time/near real-time. Inaddition, other simulated objects may be controlled by other users(e.g., located in the same or different physical location) and thechanges to attributes of the simulated object caused by control ofanother user is reflected in the simulated environment and perceived bythe user or other users using one or more devices.

One embodiment of the host server 324 includes a virtual sportssimulator 308. The virtual sports simulator 308 can be any combinationof software agents and/or hardware modules able to simulate a virtualsports game that is played by a real participant in a real worldenvironment.

The virtual sports simulator 308 is coupled to the simulator module 304and the environment simulator module 306. In one embodiment, the virtualsports simulator 308 can generate a simulated playing field thatrepresents a physical location in the real world environment. Thesimulated playing field generally has characteristics that correspond tothe physical characteristics of the physical location where the realparticipant is located. For example, if the real participant is locatedin a real park, the simulated playing field may include a grass fieldwith trees and benches. In addition, the size of the simulated playingfield can be determined based on a size of the physical location. Oneembodiment of the virtual sports simulator 308 includes a virtualplaying field generator.

The virtual sports game can be solo or team sports games. For example,the virtual sports game can be a simulation of virtual golf in adowntown square or a virtual baseball game on a crowded street corner.Even though the real street corner may not have enough room for anactual physical baseball game, the real participants can stand invarious locations with their devices (e.g., mobile devices orlocation-aware devices) and the simulated playing field canautomatically resize and readjust based on the size and othercharacteristics of the street corner in the real environment.

In one embodiment, the virtual sports simulator 308 identifies the userrequested action to be performed on a simulated object in the simulatedplaying field by detecting user interaction with the device or byreceiving data indicating the nature of the interaction of the requestedaction. In general, a simulated object in the simulated playing fieldincludes a simulated ball with a type that depends on the type of sportsof the virtual sports game. For example, the simulated ball may be agolf ball, a basketball, a baseball, a football, and/or a soccer ball.The user requested action is also typically an action to control theball that depends on the type of sports that the virtual game is.

The virtual sports simulator 308 updating a characteristic of thesimulated object in the simulated playing field according to the userrequested action and can be presented via the device such that theupdated characteristic of the simulated object is perceived by the user.The continuous or periodic updating of the simulated object and othersprovide the perception that a sporting event is occurring live. In anygiven virtual sports game, the simulated object (e.g., simulated ball)can be acted upon by multiple real participants. In addition, thevirtual sports simulator 308 may provide additional simulated objects inthe virtual sports game including but not limited to, a referee, aclock, virtual audiences, cheerleaders, living objects, animals, etc.

In one embodiment, the virtual sports simulator 308 provides a simulatedparticipant in the simulated playing field. The simulated participant istypically programmed to act as a teammate or opponent of the realparticipant. In addition, the simulated participant performs actions onthe simulated object. The actions also generally correspond to the typeof game of the virtual sports game. One embodiment of the virtual sportssimulator 308 includes a participant simulator.

The virtual sports game simulated by the virtual sports simulator 308may also be a non-competitive sports game, such as, a hike, a scubadiving session, a snorkeling session, a surfing session, etc.

One embodiment of the host server 324 includes a virtual game simulator310. The virtual game simulator 310 can be any combination of softwareagents and/or hardware modules able to simulate a virtual game that isplayed by a real participant in a real world environment. The virtualgame simulator 310 may include the gaming environment generator and theobject interaction manager module.

The virtual game simulator 310 is coupled to the simulator module 304and the environment simulator module 306. Thus, the virtual gamesimulator 310 can communicate with the modules to retrieve the simulatedobjects and/or a gaming environment to be provided to a user. Inaddition the virtual game simulator 310 can generate the gamingenvironment to a real user via a device. In general, the gamingenvironment to correspond to a physical location in the real worldenvironment where the real user is located. For example, the gamingenvironment can have characteristics that correspond to physicalcharacteristics of the physical location.

In one embodiment, the gaming environment includes a set of simulatedobjects; the accessibility of which using a device can depend on timing,location, and/or user specific parameters. For example, accessibility ofthe simulated object via the device depends on a location of the device;accessibility can further depend on the time when the device is locatedat the location. The simulated objects can include by way of example butnot limitation, reward items, ammunition, barriers, goblins, places,events, and other characters

When a simulated object in the gaming environment is accessible to areal user via a device, the real user can control the simulated objectin the gaming environment. In one embodiment, the virtual game simulator310 detects the movement of the real user and updates a characteristicof the simulated object in the gaming environment at least partiallybased on the movement of the real user.

In one embodiment, the user requested action on the simulated object inthe gaming environment can be identified by the virtual game simulator310 detecting user interactions with the device. The virtual gamesimulator 310 can thus update the characteristic of the simulated objectin the gaming environment according to the user requested action. Theupdates are typically presented through the device to be perceived bythe user and/or additional other users participating in the virtualgame.

In addition, the gaming environment can include additional simulatedobjects controlled by different real users. For example, anothersimulated object may be controlled by another real user and interactswith other simulated objects controlled by other real users in thegaming environment. Furthermore, the virtual game simulator 310 candetect the movement of another real user and updates the secondsimulated object in the gaming environment at least partially based onthe movement of the second real user. In one embodiment, the gamingenvironment includes an arcade game or a strategy game. For example, thearcade game can be a Pacman game and the real user and the second realuser control simulated objects representing Pacman. The gamingenvironment can also include other types of arcade games including butnot limited to Centipede, Frogger, etc. The strategy games can includeChess, Checkers, and/or Othello, etc.

One embodiment of the host server 324 includes a virtual performancesimulator 312. The virtual performance simulator 312 can be anycombination of software agents and/or hardware modules able to simulatea virtual performance in a real world environment.

The virtual performance simulator 312 is coupled to the simulator module304 and the environment simulator module 306. Thus, the virtualperformance simulator 312 can communicate with the modules to retrievethe simulated objects and/or a virtual performance to be provided to auser.

In one embodiment, the virtual performance simulator 312 generates asimulated object that is controlled by a real performer for display on adevice located in a physical location in the real world environment. Thereal performer may be giving a live performance in the real worldenvironment and may not necessarily be located in the physical locationwhere the simulated object is displayed on the device.

The virtual performance simulator 312 can update the simulated object inreal time or near real time according to the live performance given bythe real performer in the real world environment. The updates to thesimulated object can be presented on the device in the physicallocation, after a delayed period of time or in real time/near real time.In one embodiment, the device is suitably sized to display a full-sizeadult human being such that the simulated object of the performer can beprojected at a full size to provide the sensation of a realconcert/performance.

The simulated object can be a simulated version of the real performerhaving characteristics similar to that of the real performer. Forexample, the simulated object may have visual characteristics andresemble those of the real performer. In addition, the simulated objectmay have audible characteristics that resemble those of the realperformer. In one embodiment, the simulated object includes audio datathat is generated by the real performer during the performance. Theaudio data may also include sound effects or background music generatedin the live performance.

The live performance can be a concert where the real performer is amusician. The live performance may be a play where the real performer isan actor/actress. The live performance may be a presentation where thereal performer is a presenter.

The virtual performance simulator 312 can generate multiple simulatedobjects for display on devices located in various physical locations inthe real world environment. Each of the multiple simulated objects canrepresent the real performer giving the live performance such that thelive performance is projected at each of the multiple physical locationsin the real environment. This way, audiences, instead of having totravel to a concert, can view the simulated performance at a local ornearby location. One embodiment of the virtual performance simulator 312includes an audio module and/or a performer simulator.

One embodiment of the host server 324 includes a search engine 322. Thesearch engine 322 can be any combination of software agents and/orhardware modules able to search, detect, and/or identify simulatedobjects.

The search engine 322 can search or detect objects either automaticallyor in response to user request. For example, the user can request accessto simulated objects and perform a search request. The search requestparameters can include, one or more of, the user's location, the currenttime or a time period. The search that is performed can automaticallydetect all simulated objects that are available for access to the user.In one embodiment, the simulated objects are further filtered based onthe permissions granted to the user and/or the access permissionsassociated with the simulated object.

The host server 324 represents any one or a portion of the functionsdescribed for the modules. More or less functions can be included, inwhole or in part, without deviating from the novel art of thedisclosure.

FIG. 3B depicts an example block diagram illustrating the components ofthe host server 334 that generates and controls access to simulatedobjects.

In one embodiment, host server 334 includes a network interface 302, aprocessing unit 334, a memory unit 336, a storage unit 338, a locationsensor 340, and/or a timing module 342. Additional or less units ormodules may be included. The host server 334 can be any combination ofhardware components and/or software agents for creating, manipulating,controlling, generating simulated objects and environments. The networkinterface 302 has been described in the example of FIG. 3A.

One embodiment of the host server 334 further includes a processing unit334. The data received from the network interface 302, location sensor340, and/or the timing module 342 can be input to a processing unit 334.The location sensor 340 can include GPS receivers, RF transceiver, anoptical rangefinder, etc. The timing module 342 can include an internalclock, a connection to a time server (via NTP), an atomic clock, a GPSmaster clock, etc.

The processing unit 334 can include one or more processors, CPUs,microcontrollers, FPGAs, ASICs, DSPs, or any combination of the above.Data that is input to the host server 334 can be processed by theprocessing unit 334 and output to a display and/or output via a wired orwireless connection to an external device, such as a mobile phone, aportable device, a host or server computer by way of a communicationscomponent.

One embodiment of the host server 334 further includes a memory unit 336and a storage unit 338. The memory unit 335 and a storage unit 338 are,in some embodiments, coupled to the processing unit 334. The memory unitcan include volatile and/or non-volatile memory. In generating andcontrolling access to the simulated objects, the processing unit 334 mayperform one or more processes related to generating simulated objectsand/or controlling access to simulated objects.

In some embodiments, any portion of or all of the functions described ofthe various example modules in the host server 324 of the example ofFIG. 3A can be performed by the processing unit 334. In particular, withreference to the host server illustrated in FIG. 3A, the objectsimulator, environment simulator, access permissions functions,interactions manager functions, environmental sensing functions, objectcontrol functions, virtual sports simulator, virtual game simulator,and/or virtual performance simulator can be performed via any of thecombinations of modules in the control subsystem that are notillustrated, including, but not limited to, the processing unit 334and/or the memory unit 336.

FIG. 4A depicts an example functional block diagram of a client device402 that presents simulated objects to a user and processes interactionswith the simulated objects.

The client device 402 includes a network interface 404, a timing module406, a location sensor 408, an identification verifier module 410, anobject identifier module 412, a rendering module 414, a user stimulussensor 416, a motion/gesture sensor 418, an environmental stimulussensor 420, and/or an audio/video output module. The client device 402may be any electronic device such as the devices described inconjunction with the client devices 102A-N in the example of FIG. 1including but not limited to portable devices, a computer, a server,location-aware devices, mobile phones, PDAs, laptops, palmtops, iPhones,cover headsets, heads-up displays, helmet mounted display, head-mounteddisplay, scanned-beam display, wearable computer such as mobile enabledwatches, and/or any other mobile interfaces and viewing devices, etc.

In one embodiment, the client device 402 is coupled to a simulatedobject repository 430. The simulated object repository 430 may beinternal to or coupled to the client device 402 but the contents storedtherein can be illustrated with reference to the example of a simulatedobject repository 130 described in the example of FIG. 1.

Additional or less modules can be included without deviating from thenovel art of this disclosure. In addition, each module in the example ofFIG. 4A can include any number and combination of sub-modules, andsystems, implemented with any combination of hardware and/or softwaremodules.

The client device 402, although illustrated as comprised of distributedcomponents (physically distributed and/or functionally distributed),could be implemented as a collective element. In some embodiments, someor all of the modules, and/or the functions represented by each of themodules can be combined in any convenient or known manner. Furthermore,the functions represented by the modules can be implemented individuallyor in any combination thereof, partially or wholly, in hardware,software, or a combination of hardware and software.

In the example of FIG. 4A, the network interface 404 can be a networkingdevice that enables the client device 402 to mediate data in a networkwith an entity that is external to the host server, through any knownand/or convenient communications protocol supported by the host and theexternal entity. The network interface 404 can include one or more of anetwork adaptor card, a wireless network interface card, a router, anaccess point, a wireless router, a switch, a multilayer switch, aprotocol converter, a gateway, a bridge, bridge router, a hub, a digitalmedia receiver, and/or a repeater.

One embodiment of the client device 402 includes a timing module 406.The timing module 406 can be any combination of software agents and/orhardware modules able to identify, detect, transmit, compute, a currenttime, a time range, and/or a relative time of a request related tosimulated objects/environments.

The timing module 406 can include a local clock, timer, or a connectionto a remote time server to determine the absolute time or relative time.The timing module 406 can be implemented via any known and/or convenientmanner including but not limited to, electronic oscillator, clockoscillator, or various types of crystal oscillators.

In particular, since manipulations or access to simulated objects dependon a timing parameter, the timing module 406 can provide some or all ofthe needed timing data to authorize a request related to a simulatedobject. For example, the timing module 406 can perform the computationsto determine whether the timing data satisfies the timing parameter ofthe criteria for access or creation of a simulated object. Alternativelythe timing module 406 can provide the timing information to a hostserver to determination of whether the criteria are met.

The timing data used for comparison against the criteria can include,the time of day of a request, the date of the request, a relative timeto another event, the time of year of the request, and/or the time spanof a request or activity pertaining to simulated objects. For example,qualifying timing data may include the time the location of the device402 satisfies a particular location-based criteria.

One embodiment of the client device 402 includes a location sensor 408.The location sensor 408 can be any combination of software agents and/orhardware modules able to identify, detect, transmit, compute, a currentlocation, a previous location, a range of locations, a location at or ina certain time period, and/or a relative location of the client device402.

The location sensor 408 can include a local sensor or a connection to anexternal entity to determine the location information. The locationsensor 408 can determine location or relative location of the clientdevice 402 via any known or convenient manner including but not limitedto, GPS, cell phone tower triangulation, mesh network triangulation,relative distance from another location or device, RF signals, RFfields, optical range finders or grids, etc.

Since simulated objects and environments are associated with or haveproperties that are physical locations in the real world environment, arequest pertaining to simulated objects/environments typically includelocation data. In some instances, access permissions of simulatedobjects/environments are associated with the physical location of theclient device 402 requesting the access. Therefore, the location sensor408 can identify location data and determine whether the location datasatisfies the location parameter of the criteria. In some embodiments,the location sensor 408 provides location data to the host server (e.g.,host server 324 of FIG. 3A) for the host server to determine whether thecriteria is satisfied.

The type of location data that is sensed or derived can depend on thetype of simulated object/environment that a particular request relatesto. The types of location data that can be sensed or derived/computedand used for comparison against one or more criteria can include, by wayof example but not limitation, a current location of the client device402, a current relative location of the client device 402 to one or moreother physical locations, a location of the client device 402 at aprevious time, and/or a range of locations of the client device 402within a period of time. For example, a location criterion may besatisfied when the location of the device is at a location of a set ofqualifying locations.

One embodiment of the client device 402 includes an identificationverifier module 410. The identification verifier module 410 can be anycombination of software agents and/or hardware modules able to verify orauthenticate an identity of a user.

Typically, the user's identities are verified when they generate arequest pertaining to a simulated object/environment since somesimulated objects/environments have user permissions that may bedifferent for varying types of access. The user-specific criteria ofsimulated object access/manipulation may be used independently of or inconjunction with the timing and location parameters. The user's identitycan be verified or authenticated using any known and/or convenientmeans.

One embodiment of the client device 412 includes an object identifiermodule 406. The object identifier module 412 can be any combination ofsoftware agents and/or hardware modules able to identify, detect,retrieve, present, and/or generate simulated objects for presentation toa user.

The object identifier module 412, in one embodiment, is coupled to thetiming module 406, the location sensor 408, and/or the identificationverifier module 410. The object identifier module 412 is operable toidentify the simulated objects available for access using the device402. In addition, the object identifier module 412 is able to generatesimulated objects, for example, if qualifying location data andqualifying timing data are detected. Availability or permission toaccess can be determined based on location data (e.g., location datathat can be retrieved or received form the location sensor 408), timingdata (e.g., timing data that can be retrieved or received form thetiming module 406), and/or the user's identity (e.g., useridentification data received or retrieved from the identificationverifier module 410).

When simulated objects are available and that the access criteria aremet, the object identifier module 410 provides the simulated object forpresentation to the user via the device 402. For example, the simulatedobject may be presented via the audio/video output module 422. Sincesimulated objects may be associated with physical locations in the realworld environment, these objects may only be available to be presentedwhen the device 402 is located at or near these physical locations.Similarly, since simulated objects may be associated with real objectsin the real environment, the corresponding simulated objects may beavailable for presentation via the device 402 when near at theassociated real objects.

One embodiment of the client device 412 includes a rendering module 414.The rendering module 414 can be any combination of software agentsand/or hardware modules able to render, generate, receive, retrieve,and/or request a simulated environment in which the simulated object isprovided. The simulated environment is also provided for presentation toa user via the client device 402.

In one embodiment, the rendering module 414 also updates simulatedobjects or their associated characteristics/attributes and presents theupdated characteristics via the device 402 such that they can beperceived by an observing user. The rendering module 414 can update thecharacteristics of the simulated object in the simulated environmentaccording to external stimulus that occur in the real environmentsurrounding the device 402. The object characteristics can include byway of example but not limitation, movement, placement, visualappearance, size, color, user accessibility, how it can be interactedwith, audible characteristics, etc.

The external stimulus occurring in the real world that can affectcharacters of simulated objects can include, environmental factors in aphysical location, user stimulus, provided by the user of the device 402or another user using another device and/or at another physicallocation, motion/movement of the device 402, gesture of the user usingthe device 402. In one embodiment, the user stimulus sensor 416 receivesa request from the user to perform a requested action on a simulatedobject and can updating at least a portion of the characteristics of thesimulated object presented on the device 402 according to the effect ofthe requested action such that updates are perceived by the user. Theuser stimulus sensor 416 may determine, for example, using theidentification verifier module 410, that the user is authorized toperform the requested action before updating the simulated object.

In one embodiment, the motion/gesture sensor 418 is operable to detectmotion of the device 402. The detected motion is used by the renderingmodule 414 to adjusting a perspective of the simulated environmentpresented on the device according to the detected motion of the device.Motion detecting can include detecting velocity and/or acceleration ofthe device 402 or a gesture of the user handling the device 402. Themotion/gesture sensor 418 can include for example, an accelerometer.

In addition, based on updated locations of the device (e.g.,periodically or continuously determined by the location sensor 408and/or the rendering module 414), an updated set of simulated objectsavailable for access are identified, for example, by the objectidentifier module 412 based on the updated locations and presented foraccess via the device 402. The rendering module 414 can thus update thesimulated environment based on the updated set of simulated objectavailable for access.

The environmental stimulus sensor 420 can detect environmental factorsor changes in environmental factors surrounding the real environment inwhich the device 402 is located. Environmental factors can include,weather, temperature, topographical characters, density, surroundingbusinesses, buildings, living objects, etc. These factors or changes inthem can affect the positioning or characters of simulated objects andthe simulated environments in which they are presented to a user via thedevice 402. The environmental stimulus sensor 420 senses these factorsand provides this information to the rendering module 414 to updatesimulated objects and/or environments.

In one embodiment, the rendering module 414 generates or renders a userinterface for display on the device 402. The user interface can includea map of the physical location depicted in the simulated environment. Inone embodiment, the user interface is interactive in that the user isable to select a region on the map in the user interface. The regionthat is selected generally corresponds to a set of selected physicallocations. The object identifier module 412 can then detect thesimulated objects that are available for access in the region selectedby the user for presentation via the device 402.

The host server 402 represents any one or a portion of the functionsdescribed for the modules. More or less functions can be included, inwhole or in part, without deviating from the novel art of thedisclosure.

FIG. 4B depicts an example block diagram of the client device 402 thatpresents simulated objects to a user and facilitates user interactionswith the simulated objects.

In one embodiment, client device 402 includes a network interface 432, aprocessing unit 434, a memory unit 436, a storage unit 438, a locationsensor 440, an accelerometer/motion sensor 444, an audio outputunit/speakers 446, a display unit 450, an image capture unit 452, apointing device/sensor 454, a input device 456, and/or a touch screensensor 458. Additional or less units or modules may be included. Theclient device 402 can be any combination of hardware components and/orsoftware agents for that presenting simulated objects to a user andfacilitating user interactions with the simulated objects. The networkinterface 432 has been described in the example of FIG. 4A.

One embodiment of the client device 402 further includes a processingunit 434. The location sensor 440, motion sensor 442, and timer 444 havebeen described with reference to the example of FIG. 4A.

The processing unit 434 can include one or more processors, CPUs,microcontrollers, FPGAs, ASICs, DSPs, or any combination of the above.Data that is input to the client device 402 for example, via the imagecapture unit 452, pointing device/sensor 554, input device 456 (e.g.,keyboard), and/or the touch screen sensor 458 can be processed by theprocessing unit 434 and output to the display unit 450, audio outputunit/speakers 446 and/or output via a wired or wireless connection to anexternal device, such as a a host or server computer that generates andcontrols access to simulated objects by way of a communicationscomponent.

One embodiment of the client device 402 further includes a memory unit436 and a storage unit 438. The memory unit 436 and a storage unit 438are, in some embodiments, coupled to the processing unit 434. The memoryunit can include volatile and/or non-volatile memory. In generating andcontrolling access to the simulated objects, the processing unit 434 mayperform one or more processes related to presenting simulated objects toa user and/or facilitating user interactions with the simulated objects.

In some embodiments, any portion of or all of the functions described ofthe various example modules in the client device 402 of the example ofFIG. 4A can be performed by the processing unit 434. In particular, withreference to the client device illustrated in FIG. 4A, the timingmodule, the location sensor, the identification verifier module, theobject identifier module, the rendering module, the user stimulussensor, the motion gesture sensor, the environmental stimulus sensor,and/or the audio/video output module can be performed via any of thecombinations of modules in the control subsystem that are notillustrated, including, but not limited to, the processing unit 434and/or the memory unit 436.

FIG. 5A illustrates a diagrammatic example 500 of a simulated playingfield 504 that is provided via a device 502.

The simulated playing field 504 may be a simulation of a real lifeplaying field 506. In the example of FIG. 5A, the real life baseballfield 506 is simulated in the simulated playing field 504 and presentedvia the device 502 to the user 508.

The simulated playing field 504 is a simulated environment that includessimulated objects that correspond in features and characteristics of thereal life playing field 506. The simulated objects presented via thedevice 502 may also include interactive features and can be interactedwith or manipulated by the user 508 such that the user 508 perceives thevirtual sports game occurring in the simulated playing field 504 like asports game occurring in the real world environment 506. The user 508and other users can be real participants of the virtual sports game thatare in the real life playing field 506 of the real world environment.

FIG. 5B illustrates a diagrammatic example 510 of virtual performances514 with a simulated object 516 that is controlled by a real performer512 in a real world environment.

The virtual performances 514 are simulations of a real performanceperformed by the real performer 512, who may be giving a liveperformance in the real world environment and may be located in aphysical location distinct from the locations of the devices 502 onwhich the virtual performances 514 are presented.

The virtual performance 514 includes a simulated object 516 that iscontrolled by the real performer 512 and generally has characteristicsthat resemble those of the real performer 512. For example, thesimulated object 516 may have rendered visual features that are similarto those of the facial features of the real performer 512. In addition,the motion of the simulated object may be rendered according to themovement of the real performer 512 while giving the real performance.

In one embodiment, the simulated object 516 includes audio data that isgenerated by the real performer 512 or is synthesized based on the audiogenerated by the real performer 512 during the real performance. Theaudio data may also include sound effects or background music generatedin the live performance and/or additional simulated/synthesized sounds.The virtual performance 514 including the simulated object 516 may bepresented on a device 522 (e.g., a LCD display, a plasma display, etc.)that is suitable sized to display a fill-sized or a portion of afull-sized adult human being. The virtual performance may be presentedin real time or near real time when then live performance is being givenby the real performer 512. The virtual performance may also be presentedat a delayed time from the live performance, which may be a concert, aplay, and/or a presentation.

FIG. 5C illustrates an example screenshot on a device 502 having asimulated environment 520 with a simulated object 522 that is associatedwith a physical object 526 in a physical location in the real worldenvironment 524.

The motion, behavior, and/or action of the simulated object 522 maypartially or wholly depend on the motion, behavior, and/or action of thephysical object 526. For example, the movement of the simulated object522 in the simulated environment 520 may correspond to the movement ofthe car 526 in the real world 524. The dependence may be pre-programmedand may be re-configured by the user.

FIG. 5D illustrates a diagrammatic example of an arcade game in a gamingenvironment 530 that corresponds to a physical location in a real worldenvironment.

The arcade game shown in the example of FIG. 5D includes by way ofexample but not limitation, a Pacman game 530. The Pacman game 530 mayinclude simulated objects 532A-C that are associated with physicalobjects or real people in the real world environment. For example, thesimulated objects 532A-C may be associated with real cars on realstreets. In addition, the simulated object 532A may be controlled by areal person walking in the physical location in the real worldenvironment and the simulated object 532B may be controlled by anotherreal person. One or more of the simulated objects 532A-C may also bepartially or fully controlled (e.g., with little or no dependence onactions of real users) by a computer program.

Note that although an example is specifically provided for a Pacmangame, the arcade games that can include simulated objects that areassociated with physical objects or real people in the real worldenvironment can include other types of arcade games or strategy gamesincluding but not limited to, Centipede, Frogger, Chess, Checkers,Othello, etc.

FIG. 5E illustrates a diagrammatic example of a virtual game 540 havinga simulated combat environment that is played by a real user in a realworld environment via a device.

The virtual game 540 generally includes a simulated object (e.g.,simulated object 542) that is controlled or otherwise operated by a realuser (e.g., a user of the device on which the virtual game 540 ispresented). The virtual game 540 can optionally include a secondsimulated object (e.g., the simulated object 544), with which thesimulated object 452 interacts. The second simulated object 544 can becontrolled by another real user, a computer program, or a combination ofthe above. For example, the second simulated object 544 may be a real orsimulated opponent in the combat environment with whom the simulatedobject 542 controlled by the real user is in combat.

In general, the simulated combat environment or other virtual gamingenvironment 540 can include multiple simulated objects comprising, oneor more of reward items, ammunition, barriers, goblins, places, events,and other characters. Each object may be controlled by a real user,simulated purely in software, or a combination of the above.

FIG. 5F illustrates a diagrammatic example of a simulated objectrepresenting an interactive puzzle 550 or a component 552 thereof.

Note that although the interactive puzzle 550 is illustrated in the formas being a construction 2D puzzle, any other type of puzzle or, maze, 3Dpuzzle, or mathematical game having a digital form, can be representedby a simulated object, including but not limited to, word puzzles,jigsaw puzzles, word puzzles, tower of Hanoi, stick puzzles, tilingpuzzles, transport puzzles, and/or mechanical puzzles, etc.

In addition to the simulated object representing the puzzle, eachcomponent or some components of the puzzle 550 can be represented bysimulated objects. In the example of the construction puzzle 550 that isshown, each component 552 and/or 554 can be represented individually bysimulated objects, which may be controlled by real users and/or bysoftware programs.

FIG. 5G illustrates a diagrammatic example of simulated objects thatrepresent real-time or near-real time information/data projected ontogeographical locations in a map 560.

The example map 560 includes geographical locations spanning NorthAmerica. Note that multiple simulated objects that represent real-timeor near real-time information (e.g., information that changesdynamically), non-real time information (e.g., static information) canbe projected onto the map 560 at the relevant geographical locations. Amap spanning an entire continent and multiple countries is illustratedas an example and other types of maps having projected thereon,simulated objects that include real-time or near real-timeinformation/news or static information are contemplated. For example, amap of a state, county, city, the downtown area of a major city, aspecific neighborhood can include simulated objects associated withinformation about the location or location range, physical entities inthe location, and/or real people in the location.

For example, in the map 560, simulated object 562 is associated with NewYork City and projected at such a location in the map 560 of NorthAmerica. In this example, the simulated object 562 depicts informationand news including real time information and updates regarding theoutbreak of the Swine Flu identifying the number of confirmed cases andsuspected cases. Other relevant information can be depicted includingthe number of inbound flights and outbound flights from/to Mexico,respectively, where the outbreak of the Swine Flu is suspected to havebroken out. Other simulated objects (e.g., objects 564, 566, and 568)can be associated with other geographical locations and can depictsimilar such real time/near real time or static information/data.

In some embodiments, simulated objects are associated with real entitiesor real people (e.g., entities or people at particular geographicallocations). Such simulated objects can include information or data(which can include real time information that changes dynamically orstatic information) about the people or entities that are in or near aparticular geographical location. Such simulated objects can bespatially presented (e.g., on a 2D or 3D map) in a manner thatcorresponds to the actual physical locations of theseentities/individuals.

In some instances, the simulated objects that are associated with real(e.g., physical) entities or real people can be accessed (e.g., viewedand/or interacted with) by users via devices when the user is physicallynear or at the real entity or near the real person.

For example, when the user is near a physical store, the user may beable to access a representation of the physical store via simulatedobjects. Thus, through the virtual representation and the simulatedobjects, the user can see if it is now open, what is on sale now, howcrowded it is now, etc. Similarly, when the user is near a real person,the user may be able to access a representation (e.g., simulated object)of a real person. The representation and virtual objects can allow theuser to see various information associated with the person, includingbut not limited to, their name and profile, recent blog or microblogposts, recent photos or videos or links or documents added or annotatedby them, their recent locations, their current status or interests arefor various topics like dating, shopping, professional networking,socializing, etc.

The example map 560 of FIG. 5G includes a simulated object 570representing a drug store located in Hoboken, New Jersey. Othersimulated objects including object 572 associated with facemasks and 574associated with Tylenol in the drug store can be included. The object572 can indicate the up-to-state status on the inventory of the facemasks at the store and the object 574 can indicate that the item iscurrently on clearance sale. Potential customers can access this realtime or near real-time information, for example, before making a tripthe physical store to make their purchase to ensure that the items thatthey are interested in are either in stock and/or on sale.

Note that the simulated object 570 and the associated objects can beaccessed by the user at a remote location. In addition, the simulatedobject 570 of the drug store can be accessed by the user when they areat or near the store (e.g., within a predetermined or specified distancefrom the store).

In a further embodiment, the map 560 include a simulated object 576associated with a real-life pharmacist at the Longs Drugs in Hoboken.The simulated object 576 can include information about the real lifepharmacist, by way of example but not limitation, the name of thepharmacist (“Pharmacist Chan”), the hours that the pharmacist works,his/her position, where the degree was obtained, and/or otherspecialties. In general, the information included in the simulatedobject 576 can include real time or non-real time information.

In most instances, with various types of simulated objects thatrepresent information at a location, information related to a particularentity or individual, the user can select the type of information thatthey want to see. Note that the simulated object 576 and the associatedobjects can be accessed by the user at a remote location. In addition,the simulated object 576 associated with the pharmacist at the drugstore can be accessed and/or viewed by the user when they are at or nearthe store (e.g., within a predetermined or specified distance from thestore), or near the pharmacy section in the drug store, for example. Thesimulated object 576 that is available can be automatically detected andpresented to the user using a viewer in a device.

In general, users can specify parameters to filter the types ofsimulated objects that they would like automatically detected andpresented and types of objects that they would not like to see. Theparameters can include, by way of example, the time of day, location,distance, types of things represented by the simulated objects, theinformation contained in the simulated objects, etc.

FIG. 6 depicts a flow chart illustrating an example process fortime-based control/manipulation of a simulated object that is associatedwith a physical location in a real world environment.

In process 602, a location data and/or a timing data are determined. Inprocess 604, it is determined whether the timing data and location datasatisfy a criterion. If so, in process 606, access of the simulatedobject to a user in a simulated environment is enabled via a device. Inone embodiment, the location data includes the location of the deviceand the timing data includes a time when the device is located at thelocation. The simulated object generally includes attributes that can beperceived by the user via the device. Generally, the device is anyelectronic device including portable devices such as mobile phones,PDAs, laptop computers that may be location-aware. For example,attributes of the simulated object can include visible characteristicsand/or audible characteristics of the simulated object.

In process 608, a request is received from the user to interact with thesimulated object using the device. In process 610, it is determinedwhether the user is permitted to perform the requested action. Ingeneral, the user is associated with user access permissions. The object(simulated object) may also be associated with object accesspermissions.

If so, in process 612, the requested action is performed on thesimulated object. In process 614, the attributes of the simulated objectare updated on the device according to the requested action that isperformed to be perceived by the user using the device. Furthermore, thesimulated object can be controlled by another user located in anotherphysical location. The changes to attributes of the simulated objectcaused by control of another user can be reflected in the simulatedenvironment and perceived by the user via the device in real-time/nearreal-time, or delayed time (e.g., the changes are stored and presentedautomatically at a later time or upon request).

FIG. 7A depicts a flow chart illustrating an example process forfacilitating user interaction with a simulated object that is associatedwith a physical location in a real world environment.

In process 702, the simulated objects that are available for access areidentified based on the location data. The location data can include thelocation of a device for use by a user to access the simulated object ina time period. The simulated objects that are available for access mayfurther be identified using timing data. In process 704, an identity ofthe user is verified. In process 706, it is determined whether the useris authorized to access the simulated object. If so, in process 708, thesimulated object is provided for display on the device. In process 710,a simulated environment in which the simulated object is located isrendered. The simulated environment can be presented on the device.

In process 712, characteristics of the simulated object presented on thedevice are updated according to external stimulus that occurred in thereal environment to be perceived by the user. The external stimulus caninclude environmental factors in the physical location or user stimulusprovided by the user or other users.

In process 714, a request is received from the user to perform arequested action on the simulated object. In process 716, it isdetermined whether the user authorized to perform the requested action.In process 718, a portion of the characteristics of the simulated objectpresented on the device according to an effect of the requested actionsuch that updates are perceived by the user.

FIG. 7B depicts a flow chart illustrating example processes for updatingthe simulated object and the simulated environment according to externalstimulus.

In process 722, velocity/acceleration of the device is detected. Inprocess 714, a gesture of the user using the device is sensed. Inprocess 726, a motion of the device is detected based on the detectedvelocity/acceleration or gesture. In process 728, a perspective of thesimulated environment presented on the device is adjusted according tothe detected motion of the device.

In process 732, updated locations of the device are continuously orperiodically determined. In process 734, an updated set of simulatedobjects available for access are identified based on the updatedlocations. In process 736, the updated set of the simulated objects inthe simulated environment are presented to the user through the device.

In process 742, a user interface is rendered for display on the device.The user interface can include a map of the physical location in thesimulated environment. In process 744, a selection of a region on themap made by the user via the user interface is received. The region cancorrespond to a set of selected physical locations. In process 746, thesimulated objects that are available for access in the region selectedby the user are detected. In process 748, the simulated objects to beperceived by the user are presented via the device.

FIG. 8 depicts a flow chart illustrating an example process forsimulating a virtual sports game played by a real participant in a realworld environment.

In process 802, physical characteristics of the physical location in thereal world environment where the real participant is located areidentified. In process 804, the simulated playing field is generated fordisplay on the device. The simulated playing field generally representsa physical location in the real world environment. In one embodiment, asize of the simulated playing field is determined based on a size of thephysical location. In process 806, user interaction with the device isdetected. In process 808, a user requested action on a simulated objectin the simulated playing field is identified. The user requested actiontypically corresponds to an action that corresponds to a type of sportsof the virtual sports game and the simulated object is a simulated ballcontrolled by the user in a manner that corresponds to the type ofsports of the virtual sports game.

In process 810, a characteristic of the simulated object in thesimulated playing field is updated according to the user requestedaction. In process 812, the simulated object is presented via the deviceto such that the updated characteristic of the simulated object isperceived by the user.

In process 814, a simulated participant in provided the simulatedplaying field. The simulated participant can be programmed to act as ateammate or opponent of the real participant. The simulated participantcan also perform actions on the simulated object.

FIG. 9 depicts a flow chart illustrating an example process forsimulating a virtual game played by a real user in a real worldenvironment.

In process 902, a gaming environment is generated. The gamingenvironment corresponds to a physical location in the real worldenvironment where the real user is located. In addition, the gamingenvironment includes characteristics that correspond to the physicalcharacteristics of the physical location and includes simulated objectsthat can be controlled by the real user. In process 904, a gamingenvironment is provided to the real user via the device.

In process 906, movement of the real user is detected. In process 908, acharacteristic of the simulated object is updated in the gamingenvironment at least partially based on the movement of the real user.In general, the accessibility of the simulated object via the devicedepends on the location of the device and/or a time or time range whenthe device is located at the location.

In process 910, user interaction with the device is detected. In process912, a user requested action on the simulated object in the gamingenvironment is identified. In process 914, the simulated object isupdated in the gaming environment according to the user requestedaction. In process 916, movement of a second real user is detected. Inprocess 918, the second simulated object is updated in the virtualgaming environment at least partially based on the movement of thesecond real user. The second simulated object may interact with thesimulated object in the gaming environment.

In general, the gaming environment includes multiple simulated objectsincluding but not limited to, reward items, ammunition, barriers,goblins, places, events, and/or other characters. The requested useraction with the simulated object can include, collecting a reward item,firing ammunition, throwing an item, consuming an item, attending anevent, dialoguing with another character, surmounting a barrier, and/orshooting a goblin.

FIG. 10 a flow chart illustrating an example process for simulating avirtual performance in a real world environment.

In process 1002, a simulated object is generated for display on a devicelocated in the physical location in the real world environment. In oneembodiment, the simulated object is controlled by a real performergiving a live performance in the real world environment. The realperformer may or may not be necessarily located in the physical locationwhere the simulated object is displayed on the device.

In process 1004, the live performance given by the real performer ismonitored. In process 1006, the simulated object is updated in real timeor near real time according to the live performance. Alternatively, thesimulated object can be updated after a delay (e.g., the updates can bestored and presented at a later time). The real performer may be amusician, an actor/actress, and/or a presenter.

In process 1008, updates to the simulated object are presented on thedevice in the physical location. Note that the device can be a portabledevice or suitably sized to display a full-size adult human being. Thesimulated object can include audio data generated by the real performeror sound effects/background music generated in the live performance.

In process 1010, multiple simulated objects are generated for display ondevices located in multiple physical locations. Each of the multiplesimulated objects represent the real performer giving the liveperformance such that the live performance is projected at each of themultiple physical locations in the real world environment.

FIG. 11 shows a diagrammatic representation of a machine in the exampleform of a computer system 1100 within which a set of instructions, forcausing the machine to perform any one or more of the methodologiesdiscussed herein, may be executed.

In alternative embodiments, the machine operates as a standalone deviceor may be connected (e.g., networked) to other machines. In a networkeddeployment, the machine may operate in the capacity of a server or aclient machine in a client-server network environment, or as a peermachine in a peer-to-peer (or distributed) network environment.

The machine may be a server computer, a client computer, a personalcomputer (PC), a tablet PC, a set-top box (STB), a personal digitalassistant (PDA), a cellular telephone, a web appliance, a networkrouter, switch or bridge, or any machine capable of executing a set ofinstructions (sequential or otherwise) that specify actions to be takenby that machine.

While the machine-readable medium is shown in an exemplary embodiment tobe a single medium, the term “machine-readable medium” should be takento include a single medium or multiple media (e.g., a centralized ordistributed database, and/or associated caches and servers) that storethe one or more sets of instructions. The term “machine-readable medium”shall also be taken to include any medium that is capable of storing,encoding or carrying a set of instructions for execution by the machineand that cause the machine to perform any one or more of themethodologies of the present invention.

In general, the routines executed to implement the embodiments of thedisclosure, may be implemented as part of an operating system or aspecific application, component, program, object, module or sequence ofinstructions referred to as “computer programs.” The computer programstypically comprise one or more instructions set at various times invarious memory and storage devices in a computer, and that, when readand executed by one or more processors in a computer, cause the computerto perform operations to execute elements involving the various aspectsof the disclosure.

Moreover, while embodiments have been described in the context of fullyfunctioning computers and computer systems, those skilled in the artwill appreciate that the various embodiments are capable of beingdistributed as a program product in a variety of forms, and that thedisclosure applies equally regardless of the particular type of machineor computer-readable media used to actually effect the distribution.

Further examples of machine or computer-readable media include but arenot limited to recordable type media such as volatile and non-volatilememory devices, floppy and other removable disks, hard disk drives,optical disks (e.g., Compact Disk Read-Only Memory (CD ROMS), DigitalVersatile Disks, (DVDs), etc.), among others, and transmission typemedia such as digital and analog communication links.

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise,” “comprising,” and thelike are to be construed in an inclusive sense, as opposed to anexclusive or exhaustive sense; that is to say, in the sense of“including, but not limited to.” As used herein, the terms “connected,”“coupled,” or any variant thereof, means any connection or coupling,either direct or indirect, between two or more elements; the coupling ofconnection between the elements can be physical, logical, or acombination thereof Additionally, the words “herein,” “above,” “below,”and words of similar import, when used in this application, shall referto this application as a whole and not to any particular portions ofthis application. Where the context permits, words in the above DetailedDescription using the singular or plural number may also include theplural or singular number respectively. The word “or,” in reference to alist of two or more items, covers all of the following interpretationsof the word: any of the items in the list, all of the items in the list,and any combination of the items in the list.

The above detailed description of embodiments of the disclosure is notintended to be exhaustive or to limit the teachings to the precise formdisclosed above. While specific embodiments of, and examples for, thedisclosure are described above for illustrative purposes, variousequivalent modifications are possible within the scope of thedisclosure, as those skilled in the relevant art will recognize. Forexample, while processes or blocks are presented in a given order,alternative embodiments may perform routines having steps, or employsystems having blocks, in a different order, and some processes orblocks may be deleted, moved, added, subdivided, combined, and/ormodified to provide alternative or subcombinations. Each of theseprocesses or blocks may be implemented in a variety of different ways.Also, while processes or blocks are at times shown as being performed inseries, these processes or blocks may instead be performed in parallel,or may be performed at different times. Further any specific numbersnoted herein are only examples: alternative implementations may employdiffering values or ranges.

The teachings of the disclosure provided herein can be applied to othersystems, not necessarily the system described above. The elements andacts of the various embodiments described above can be combined toprovide further embodiments.

Any patents and applications and other references noted above, includingany that may be listed in accompanying filing papers, are incorporatedherein by reference. Aspects of the disclosure can be modified, ifnecessary, to employ the systems, functions, and concepts of the variousreferences described above to provide yet further embodiments of thedisclosure.

These and other changes can be made to the disclosure in light of theabove Detailed Description. While the above description describescertain embodiments of the disclosure, and describes the best modecontemplated, no matter how detailed the above appears in text, theteachings can be practiced in many ways. Details of the system may varyconsiderably in its implementation details, while still beingencompassed by the subject matter disclosed herein. As noted above,particular terminology used when describing certain features or aspectsof the disclosure should not be taken to imply that the terminology isbeing redefined herein to be restricted to any specific characteristics,features, or aspects of the disclosure with which that terminology isassociated. In general, the terms used in the following claims shouldnot be construed to limit the disclosure to the specific embodimentsdisclosed in the specification, unless the above Detailed Descriptionsection explicitly defines such terms. Accordingly, the actual scope ofthe disclosure encompasses not only the disclosed embodiments, but alsoall equivalent ways of practicing or implementing the disclosure underthe claims.

While certain aspects of the disclosure are presented below in certainclaim forms, the inventors contemplate the various aspects of thedisclosure in any number of claim forms. For example, while only oneaspect of the disclosure is recited as a means-plus-function claim under35 U.S.C. § 112, ¶6, other aspects may likewise be embodied as ameans-plus-function claim, or in other forms, such as being embodied ina computer-readable medium. (Any claims intended to be treated under 35U.S.C. § 112, ¶6 will begin with the words “means for”.) Accordingly,the applicant reserves the right to add additional claims after filingthe application to pursue such additional claim forms for other aspectsof the disclosure.

What is claimed is:
 1. A method of facilitating interaction, via a device, with a simulated object that is associated with a physical object at a physical location in a real environment, the method comprising: obtaining a location data of the device while displaying images obtained by a camera of the device; in response to determining that the location data obtained by the device satisfies a criterion specifying where the simulated object can be generated, providing the simulated object with the images obtained by the camera; and in response to detecting a motion of the device, automatically adjusting the simulated object according to the motion of the device.
 2. The method of claim 1, further comprising: detecting an external stimulus occurring in the real environment surrounding the device, wherein the external stimulus comprises environmental factors of the real environment.
 3. The method of claim 2, further comprising: detecting whether the external stimulus is changed; and in response to detecting whether the external stimulus is changed, adjusting the simulated object.
 4. The method of claim 3, further comprising: in response to detecting whether the external stimulus is changed, providing another simulated object with the images obtained by the camera.
 5. The method of claim 1, wherein the simulated object comprising at least one indication for indicating a destination of the device.
 6. The method of claim 1, further comprising: providing an user interface including a map while providing the simulated object.
 7. The method of claim 1, further comprising: continuously or periodically determining updated locations of the device; identifying an updated set of simulated objects available for access based on the updated locations; and accordingly presenting the updated set of simulated objects in a simulated environment.
 8. The method of claim 1, wherein the device comprises head-mounted display device.
 9. A device comprising: a location sensor; a camera; a display; and a processor configured to: obtain a location data of the device while displaying images obtained by the camera, in response to determining that the location data satisfies a criterion specifying where a simulated object can be generated, providing the simulated object with the images obtained by the camera, and in response to detecting a motion of the device, automatically adjust the simulated object according to the motion of the device.
 10. The device of claim 9, wherein the processor is further configured to: detect an external stimulus occurring in a real environment surrounding the device, and wherein the external stimulus comprises environmental factors of the real environment.
 11. The device of claim 9, wherein the processor is further configured to: detect whether the external stimulus is changed, and in response to detecting whether the external stimulus is changed, adjusting the simulated object.
 12. The device of claim 11, wherein the processor is further configured to: in response to detecting whether the external stimulus is changed, provide another simulated object with the images obtained by the camera.
 13. The device of claim 9, wherein the simulated object comprising at least one indication for indicating a destination of the device.
 14. The device of claim 9, wherein the processor is further configured to: provide an user interface including a map while providing the simulated object.
 15. The device of claim 9, wherein the processor is further configured to: determine updated locations of the device, identify an updated set of simulated objects available for access based on the updated locations, and providing the updated set of simulated objects.
 16. The device of claim 9, wherein the device comprises head-mounted display device.
 17. A method of facilitating interaction, via a device, with a simulated object that is associated with a physical object at a physical location in a real environment, the method comprising: in response to determining that timing data obtained by the device satisfies a criterion specifying when the simulated object can be generated, providing the simulated object for presentation via the device, the presentation of the simulated object being based at least in part on a perceptible characteristic of the physical object; rendering a simulated environment in which the simulated object is provided, the simulated environment presented via the device; in response to detecting a motion of the device, automatically adjusting a perspective of the simulated environment presented on the device according to the motion of the device; based on the adjusting of the perspective, providing the simulated environment with at least one new simulated object of simulated objects associated with a location of the device where the device has access permission to the at least one new simulated object at the location of the device; detecting changes occurring in the real environment while rendering the simulated environment; and in response to detecting the changes occurring in the real environment, adjusting the simulated environment to depict the changes occurring in the real environment, wherein the timing data comprises a time when the device is at the physical location.
 18. The method of claim 17, further comprising: detecting an external stimulus occurring in the real environment surrounding the device, wherein the external stimulus comprises environmental factors of the real environment comprising buildings.
 19. The method of claim 18, wherein the external stimulus further comprises a user stimulus provided by a user of the device or another user.
 20. The method of claim 17, wherein the simulated object is determined to be available for access based on the timing data or location data associated with the device. 